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Higher Education Accountability for Student Learning* 

Summary 

Each year, states spend collectively more than $70 billion to support higher education, but 
governors and the public do not have accessible, useful information about what students learn as a 
result of their time in college. Without reliable information about postsecondary learning 
outcomes, policymakers can not determine which investments or strategies are most cost- 
effective, and students, families, and employers do not have information that can improve their 
decisions about the quality of different providers of higher education. 

Governors can help restrain college costs — while extending a quality postsecondary education to 
a larger segment of the population — by insisting that student learning outcomes become an 
integral part of state higher education accountability systems. Governors can help build 
accountability systems that distinguish between state-level and institutional standards and 
assessments, while providing support for each. State-level accountability systems need to focus 
on aggregate statewide objectives for postsecondary education, whereas institutional systems can 
reflect institution-specific priorities. Specifically, governors can: 

• Call for the development of minimum general educational learning outcomes for 
undergraduates educated at a public college or university, and require assessment of these 
outcomes. There is already a good deal of consensus among institutions about these 
goals; gubernatorial involvement can help elevate this consensus into a stronger tool for 
public accountability and enhanced institutional performance. These learning objectives 
may be stated as a set of common undergraduate competencies that all students must 
demonstrate, such as strong analytical, communication, quantitative, and information 
skills. 

• Require student competencies to be assessed and publicly reported through appropriate 
metrics, such as a combination of statewide sampling and institutional assessments. 

Reasons for State-Level Student Learning Assessment 

Traditionally, policymakers have measured performance in higher education by looking at 
institutional information on inputs (e.g., funding per student, admissions standards) and activity 
measures (e.g., enrollment, retention, time-to-degree). Today’s focus on learning represents a 
major evolution in state higher education policy. Governors, legislators, and higher education 
leaders want to know how resources are used to produce knowledge. To do this they need ways to 
measure learning outputs, taking into account differences in institutional mission and different 
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levels of student preparation. There are many reasons for governors’ increased interest in using 
the outcome data from student learning assessments. 

The economic imperative of expanding college access and achievement. The combined effects 
of upcoming hahy-hoom retirements, a rapidly changing demographic profile in the states, and 
restructuring in favor of skilled jobs suggests there may not be enough college-educated workers 
in the future. Sixty-seven percent of new jobs created by 2010 are projected to demand skills that 
require at least some college, leaving the nation short at least 10 million workers with 
postsecondary education. This workforce shortage can not be filled without an expansion of 
college access and achievement, particularly since two-thirds of 1 8 to 24 year olds will be from 
ethnic and income groups historically underrepresented in postsecondary education. ' 

The United States’ system of higher education has been reasonably successful in providing 
growing levels of access to higher education. Since 1972, college-going rates for high school 
graduates have grown from 58 percent to 72 percent.^ Gains in college-going have occurred 
among all ethnic groups, although the access gaps between groups remain. Success in expanding 
access has not been paralleled with comparable gains in degree attainment: currently, 54 percent 
of undergraduate students persist to a baccalaureate degree within six years. The ethnic 
achievement gaps widen again when it comes to persistence and degree attainment: White and 
Asian baccalaureate attainment rates are 20 or more percentage points higher than for Hispanics 
or African-Americans.^ 

Student portability . Students are increasingly mobile and accumulate credits toward degrees from 
several institutions. Nationwide, over 20 percent of students who received the baccalaureate 
degree in 1992 had taken classes from at least three different institutions."^ This means 
institutional measures of retention and graduation are less relevant than statewide or regional 
measures that capture learning across several institutions. Growing student mobility means 
quality control mechanisms also must evolve, from the institution and course as the unit of 
analysis to broad student learning across institutions. 

Learning Outcomes and Regional Accreditation 

For the last decade, institutional and specialized accreditation agencies have made student 
learning assessment and improvement a central focus of accreditation review. The increased 
emphasis on learning has come about in part because of federal mandates requiring all accreditors 
seeking federal recognition to focus on learning assessment in addition to other measures of 
quality. As part of its collaborative work with the Business-Higher Education Forum’s project on 
student learning outcomes, in 2004 the National Governors Association (NGA) commissioned 
University of Maryland researchers Ben and Susan Passmore to conduct a survey of two- and 
four-year regionally accredited degree-granting institutions to assess the status of learning 
outcome assessments. They reviewed institutional statements of learning outcomes to identify 
any commonalities or patterns across different types of institutions. They also looked to see how 
the goals were assessed, although they did not request information about public communication 
of results. The major findings from that survey are: 

• Virtually all institutions either have (78 percent) or are developing (16 percent) 

explicit statements of learning goals. These are made public through publication in 
course catalogues in planning documents, and as part of public accountability reports. 
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Roughly 20 percent of the institutions had explicit suhject-area goals; the rest had 
general goals for hroad learning outcomes. 

• A smaller proportion of institutions (55 percent, including 15 percent who are still 
working on their assessment plans) have institution-level assessments in place to 
evaluate whether the general learning goals are met. The remainder leave 
assessments of learning to the faculty as part of course-level evaluations. 

• There is a considerable amount of consistency among different types of institutions 
about learning outcomes expected of graduates: communication, critical thinking, and 
numeracy skills; computer/information technology skills; and ethics. Additionally, 
more than half of the institutions state learning goals in terms of broad mastery of the 
subject areas of fine arts, humanities, social and natural sciences. 

The results seem to indicate there is a good deal of latent consensus about expectations for broad 
learning outcomes for undergraduate study. There is also evidence that institutions are serious 
about the assessment of student learning. However, in many institutions the assessments are not 
routinely linked back to broad goals for learning, and many learning goals are framed as 
curricular expectations for general education rather than explicit knowledge and skill outcomes. 

Models for Assessing Student Learning Outcomes in Higher Education at the State 
Level 

Embedding student learning measures into statewide accountability systems will require different 
models than are in place for K-12 education. State K-12 accountability systems are built upon a 
model that tests all students through a single, state-level examination. School and district 
performance is based on changes from year-to-year in student achievement. Comparisons 
between schools are possible because all students are evaluated using the same test of proficiency 
on statewide learning standards. 

In higher education, institutional diversity and decentralized governance contributes to a lack of 
standard curriculum or single set of learning standards for all students in all institutions. In 
addition to diverse curricula, large differences exist between institutions in admissions practices 
and in the academic skills and preparation level of incoming students. 

In the last decade, regional accreditors have started to require institutions to do much more to 
measure student learning results, and as a result every institution now has some policy on student 
learning assessments. These assessments are essentially instruments for campus improvement and 
internal institutional accountability. They are not designed to give policymakers or consumers 
information that would allow them to make comparisons between institutions. Nonetheless, the 
work that has already been done by institutions, working with regional accreditors, provides a 
good starting place for taking learning outcome assessment to the next level of being better linked 
to public accountability systems. 

The diversity of assessments does not mean assessment-linked public accountability is impossible 
in higher education; however, it does mean different techniques need to be developed than are in 
place in K-12. Innovative states such as Georgia, Kentucky, South Dakota, Tennessee, Texas, 
and Virginia are using different models to embed student learning outcome data in state higher 
education accountability systems. These include: 

• aggregating data from nationally-normed, external examinations; 
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• surveying of the postsecondary learning experience; 

• creating a state mandate that institutions select, administer, and publicly report student 
performance data; and 

• requiring common statewide examinations for all public colleges and universities. 
Aggregating Data from Nationally-Normed, External Examinations 

One approach states can take is to aggregate existing assessments from professional and graduate 
school entrance exams and licensing examinations. This approach has the advantage of being 
inexpensive, since the states do not have to pay to develop or administer the examinations. Many 
college graduates already sit for some kind of externally-administered, nationally-normed 
examination, such as the Graduate Entrance Examination; licensure examinations required for 
certain professions; or professional school exams such as the LSAT and MCAT. These results 
cannot be generalized to describe reliably what all students enrolled in a public college or 
university know or have learned as a result of their postsecondary experiences because these 
examinations are not required of all students or even of all graduates. Nonetheless, they do offer 
governors a snapshot of assessed knowledge and skill levels for this sub-section of college 
graduates. 

Another option being tried by some states is to participate in the administration of the national 
assessments of adult literacy (NAAL). Administered by the National Center for Education 
Statistics, NAAL is an examination of adult literacy and numeracy skills administered to a 
nationally representative sample of American adults aged 16 and older. NALS provides national 
snapshots of broad changes in adult literacy. Because the test is not confined to the skill levels of 
college attendees or graduates, it is not a proxy for measuring the results of college learning. 
However, it can provide states with useful information about general skill levels among their 
adult populations, which can contribute to state-level planning and evaluation. This approach is 
being used in several states - among them. New York, Kentucky, Maryland, and 
Massachusetts. 

Surveys of the college learning experience. In addition to direct assessments of learning, states 
can indirectly measure student learning by assessing a reliable proxy, the college learning 
experience. Student experience surveys are increasingly common in postsecondary education. 

The externally-developed National Survey of Student Engagement (NSSE) and Community 
College Survey of Student Engagement (CCSSE) annually survey samples of students and recent 
graduates from participating four- and two-year colleges. Students are surveyed on such 
experiences as academic challenge, time spent in class and preparing for class, and interaction 
with faculty and peers. Research shows qualitative learning experiences such as these contribute 
to college learning in all institutions. CCSSE requires community college results to be made 
public; publication of NSSE results remains an institutional decision. Despite these limitations, 
surveys of college learning experiences can help institutions improve student learning outcomes 
by paying attention to good practices in teaching and learning. 

A state mandate to select, administer, and report student learning outcomes. The most common 
approach to state-based assessment of student learning is a statewide mandate to institutions to 
conduct some form of regular assessment of student learning. These mandates give institutions 
the discretion to select and administer assessments independently. This model connects to 
requirements of the accreditation process and does not require duplicate and expensive 
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assessments. It defers primary responsibility for setting goals and measuring results to the 
institution and does not produce comparative data that can be aggregated at the statewide level. 
However, many states additionally require the institutions to use peer group data to develop their 
own benchmarks for comparative analysis of learning. Peers are selected based on comparability 
of mission, programs, funding, and student admissions and are subject to state-level review and 
approval. This model has the advantage of respecting institutional autonomy and decentralization 
of policy responsibility for learning assessments. It also provides institutional leaders with data 
with which to evaluate their performance over time, relative to their peer group. By itself, this 
model does not help to focus on statewide rather than institutional accountability because of the 
focus on mission-specific performance. 

Common statewide examinations for all public colleges and universities. A handful of states 
are requiring statewide, institutionally-based, student testing on a common examination — the 
closest analogy to the K-12 accountability model in higher education. Some states use nationally- 
developed examinations of basic skills for these purposes; others have developed their own; and 
some use a combination of home-grown and commercial instruments. The typical model in place 
now requires assessments of basic skills before students are allowed to progress to junior status or 
prior to degree attainment. For instance, several states, including South Dakota and Tennessee, 
require “rising juniors” to pass a nationally -developed examination of basic skills. Georgia and 
Florida have developed their own state-based examinations of basic skills. Georgia’s is the 
Regents’ Testing Program (RTP), which is administered to all students in public degree-granting 
institutions at the sophomore or junior level. Students must show they have passed the 
examination to obtain their degrees. The Texas Success Initiative consists of a set of state-level 
standards that reference numerous college placement tests. The tests evaluate students’ level of 
preparation in basic reading, writing, and math. Institutions use these standards to determine 
whether students are considered prepared for college-level courses.^ 

Characteristics of Effective State-Level Assessment of Student Learning Outcomes 

It is too early to judge whether state-based learning assessments accomplish the goals of 
improving statewide performance in student learning. These systems are relatively young, and 
many have been controversial. They are being subjected to constant evaluation and much 
tinkering. Many have been implemented at a time of real fiscal crisis in the states, when tuition 
increases and budget cutbacks were severely affecting student access and course availability. 
These factors make it difficult to discern the consequences of learning assessments as distinct 
from the other pressures on higher education institutions. 

Despite their newness, experiences from many states are beginning to yield common wisdom 
about how student learning outcomes can be integrated into statewide accountability systems to 
help policymakers and institutional leaders remain focused on improving teaching and learning. 

Shared Responsibility for Accountability: State-Level and Institutional. Accountability systems 
work best when they support state-level as well as institutional accountability and differentiate 
clearly between the two in assessments and audience. The primary audience for state-level 
accountability for learning assessments is state policy makers — governors and legislators — 
supported by statewide governing or coordinating boards. State policymakers need to be able to 
see aggregate cross-sector measures of student learning to help keep them focused on statewide 
progress in meeting learning goals, as part of their responsibilities for planning and policy 
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development. Institutional assessment and accountability structures serve the dual purpose of 
providing better public information about purpose and performance and the support of 
institutional improvements in teaching and learning. Kentucky and South Dakota integrate state- 
level and institutional efforts to improve student learning outcomes with a few clear, statewide 
goals and measures. Kentucky has a statewide accountability system that directly reinforces its 
postsecondary education reform legislation and ties it to the governor’s goal for higher 
education — increasing educational attainment to the national average in order to increase median 
family income to the national average. 

Kentucky and its institutions measure progress on this long-term goal by focusing attention on its 
strategic plan or public agenda — five key questions related to preparation, affordability, 
participation, learning, and economic and community engagement. Each of these questions and 
an overall accountability report card for the state is presented annually to the governor, 
legislature, and public. Student learning outcomes are aggregated in this accountability system, 
using sampling data from the NSSE and CCSSE student engagement surveys, the NAES, and 
licensure and certification exams. Kentucky’s statewide coordinating agency, the Council for 
Postsecondary Education, is currently reviewing what kinds of institution-specific data on student 
learning are needed and has recurring state funding to support its efforts. Kentucky also 
consented to being the first pilot state to report aggregated student learning outcome data in the 
National Center for Public Policy and Higher Education’s Measuring Up 2002 report card and 
participate in the five state pilot project in 2004. Using the report card’s benchmarking system, 
Kentucky policymakers could determine: 

■ Verbal literacy levels for Kentucky’s college-educated residents are better than 
average, but the state remains below the nation in quantitative literacy levels. 

■ Kentucky’s higher education outcomes and good practices are only average. 

■ Kentucky’s postsecondary institutions contribute more to vocational and 
professional preparation than preparation for graduate education. 

Similarly, South Dakota has four state higher education policy goals. One of the state’s goals is 
for South Dakota public universities and special schools to provide a quality educational 
experience. At the state-level, all rising juniors in the system take ACT’s College Assessment of 
Academic Proficiency (CAAP) to determine students’ mathematical, scientifiic reasoning, 
writing, and reading skills. The purpose of the examination is to ensure students are making 
expected improvements over their entering ACT score. The South Dakota Board of Regents 
receives and publicizes this comparable information about institutional “value-add.” 

At the institution-level, the Board of Regents requires every institution to select a benchmark for 
student learning outcomes and an appropriate assessment instrument. These data are reported 
publicly, and 20 percent of the state’s institutional incentive funds are based on performance 
against this benchmark. Black Hills State University, for example, reports the average 
mathematics score on the CAAP; the University of South Dakota reports the percentage of 
students enrolled in courses that require significant research or creative activity; and Northern 
State University reports the percentage of bachelor’s graduates completing an e-learning 
certificate. 

The Board of Regents is currently reviewing student learning outcome data, system policies for 
associate and baccalaureate degree general education, and specific general education course 
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requirements. Possible outcomes include increasing the general education mathematics 
requirement and increasing the writing expected of students.® 

Connections: Good statewide assessment and accountability systems reinforce connections 
between institutions, including the pipeline between higher education and K-12. They keep the 
focus on student access and flow across institutions and give institutional and state policymakers 
tools to identify and address achievement gaps. They also can be vehicles for aligning high school 
graduation requirements with expectation for college admissions and placements. 

Georgia’s postsecondary assessment programs directly support its statewide higher education 
accountability system and P-12 educational pipeline. The Regents for the University System of 
Georgia, who set policy for all 34 state-supported two-year colleges and universities, established 
a P-16 Initiatives office to help coordinate between and among other state educational agencies. 
The state is currently developing a P-16 database that will enable school districts to follow 
students from preschool through college graduation, including data on postsecondary learning. 

The Regents are focusing on closing achievement gaps in retention and six-year graduation rates. 
They have recently developed policies on institutional practices that are found to be effective in 
closing gaps and increasing overall completion rates. These practices resulted from a task force 
established to develop a five-year plan to bring University System six-year graduation rates at 
least to the national average. Presently, the system average is over 10 percentage points below the 
national average. Annually, institutions will be required to report disaggregated retention rates for 
years one through six. Ah institutions will administer either NSSE or CCSSE as a gauge of 
student engagement. In addition, institutions are being encouraged to work with their major 
feeder high schools, in particular through the local P-16 councils, to evaluate postsecondary 
performance. 

Virginia has the foundation for making comparisons across institutions with six statewide 
competency-based assessments. These assessments — writing, technological literacy, quantitative 
reasoning, scientific reasoning, critical thinking, and oral communications — are a product of the 
1999 Governor’s Blue Ribbon Commission on Higher Education. These core competencies do 
not attempt to certify individual graduates (the degree awarded does this). Instead, the 
competencies validate that students learn what institutions purport to teach. 

The first round of results of the two initial competencies — writing and technological literacy — 
was reported to the State Council of Higher Education for Virginia (SCHEV) and included in the 
state’s public report card for higher education, known as the Reports on Institutional 
Effectiveness. Competency assessment approaches (e.g., methods, sample sizes, and evaluation 
instruments) vary widely across institutions. SCHEV works cohaboratively with institutions to 
provide feedback for improvements in implementation, measurement, and reporting criteria. 
SCHEV also facilitates efforts to increase the rigor associated with competency assessments. At 
the state level, these student learning outcome data inform statewide policies on articulation, 
transfer, and academic integrity. 

Virginia’s statewide core competencies also provide a common framework for institutional-level 
assessments. These assessments are tailored to suit the needs of each institution, consistent with 
its mission and student population. Eor instance, at James Madison University, ah entering 
freshmen are given general education assessments in the six state competency areas, plus 
government and wellness. Ah entering freshmen are assessed prior to matriculation, and then 
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again after two years. All students must demonstrate general education competencies before 
being allowed to continue tbeir education. Tbe results are also used to re-evaluate the curriculum: 
every general education course sequence must regularly demonstrate positive learning results to 
remain in the core curriculum.’ 

Context and comparisons: For data to make sense, it has to be put into context through the 
use of comparison information and by being monitored over time. 

The most effective assessment and accountability systems use comparison information - either 
from other states, or at the institutional level. Lawmakers and institutional leaders well 
understand the uniqueness of their circumstances and know the limitations of data that inevitably 
make unfair comparisons. 

In Texas, the higher education coordinating board has set four goals and related performance 
targets for higher education-student participation, student success, institutional excellence, and 
research. These performance areas serve as the basis for a governor-mandated accountability 
system. The accountability system consists of a small number of key measures and a few 
contextual variables for each performance area. Similar institutions are partnered to determine 
key measure improvement targets and share best practices that have enhanced performance. 
Learning outcomes are not explicitly part of the current system but are included in discussion of 
future directions. 

Within this state framework, the University of Texas (UT) system has implemented a centralized, 
consistent, and detailed accountability framework. The report includes 25 undergraduate 
participation and success measures for its nine institutions; each measure is displayed with five- 
year longitudinal data, and comparisons are made to institution goals, rather than institution to 
institution. ^ One-third of these undergraduate performance indicators directly or indirectly 
measure student learning outcomes, including licensure and certification exam pass rates, and 
student satisfaction measures. In the future, the UT system plans to incorporate results from the 
NSSE and the Council for Aid to Education’s undergraduate value-added assessment instrument.® 

Infrastructure: The development and maintenance of an assessment and accountability system 
to measure success. 

The best systems are embedded within a larger structure of shared governance and mutual 
accountability between the governor and the legislature, the statewide governing or coordinating 
board, and institutional leaders. The systems need to be capable of withstanding changes in 
leadership to ensure they can be sustained over time. If the measures are in a constant state of 
redefinition, no coherent trend line ever emerges. The measures need to have some credibility 
with the different audiences expected to use the information. Absolute consensus is not 
imperative, but institutional buy-in is necessary. 



Tennessee’s 20-year-old accountability system, the Performance Eunding Program, is designed to 
stimulate instructional improvement and student learning. It provides the governor, legislature, 
and public the means of assessing progress of publicly funding higher education. While the 
governor and legislator play oversight roles in the system, institutional and system representatives 
are directly involved in the development of performance standards. 
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The Performance Funding Program provides institutions with supplemental, discretionary 
resources for desired outcomes including the assessment of undergraduate student learning. 
Through the various assessment initiatives, policymakers are able to gauge the effectiveness of 
instruction as well as the quality of student services. Since Performance Funding was initiated, all 
institutions have accepted methods of assessing general education (only two public institutions 
engaged in this activity prior to performance funding). Recently, the indicator was revised to 
encourage campuses to show how they use assessment data in campus decisions. Case study 
evaluations cite strong “ownership” of the program by campus and government officials. The 
state continues to grapple, however, with how to improve student motivation to take assessments 
when the results do not serve as a barrier to further study or graduation. 



Recommendations for Governors 

Gubernatorial leadership is essential to the management of effective and sustainable higher 
education accountability systems. Governors are in a unique position to set the agenda by 
defining the terms of the conversation, bringing stakeholders together in a process, and getting 
buy-in from institutional and sector governing boards. They can move the agenda by developing 
short-term goals that produce visible results, and at the same time embedding the system within 
durable, on-going processes. Some specific recommendations can help move this forward. 

Call for the identification of statewide learning outcomes for undergraduates educated at a 
public college or university and require assessment of these common learning objectives. 
Governors can start by calling for statewide attention on the need to ensure improvements in 
student learning and framing subsequent discussions in terms of statewide goals and 
accountability strategies. In the short-term, governors can establish a blue ribbon commission for 
accountability for student learning in higher education and charge it with the responsibility to 
identify a discrete set of common undergraduate competencies that all college students must 
demonstrate, such as strong analytical, communication, quantitative, and information skills. If the 
existing statewide coordinating or governing board is up to the task, it can be charged with this 
responsibility instead. The key is to have a process that involves a balance of external and internal 
stakeholders, including institutional representatives and leaders from K-12 education, businesses, 
and communities. The group should be given a short timeline within which to do its work — six 
months to one year. 

In the mid-term, governors can ask each of the public governing boards to show how they are 
attending to assessment and accountability within their institutions. Governors can ask the 
boards’ leadership to participate in an annual meeting to talk about their internal assessment and 
accountability efforts. Governors can also ask institutions to build the data capacity to track 
students across institutions, including out-of-state institutions. If necessary, governors can support 
the development of new statewide assessment tools designed to measure state- specific goals. 

And finally, governors can request an evaluation of the accountability system every five years to 
learn what is working and what needs to be changed. 

Require student competencies to be assessed through statewide sampling and on an 
institutional basis through assessments embedded in general education and departmental 
major courses. 
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Governors can support the need to combine institutional, sample, or direct assessments of 
learning to measure progress in meeting statewide learning goals. 

In the short-term, governors can ask each institution to build on the work it has already done in 
setting learning goals by documenting how it is assessing its learning goals. Further, governors 
can ask institutions to develop benchmarks for comparisons of learning performance with 
institutions in- and out-of-state. Additionally, governors can encourage institutions to participate 
in the NSSE, CCSSE, and NAES surveys and share their results at a state level. Eurther, 
governors should require public report cards to include comparable student learning outcome 
data, just as they include comparable data about the incoming freshman class, or the six-year 
degree completion rate of first-time, full-time students. 



Use student learning outcome data in state-level decision-making about resources and program 
approval 

Governors should ground every discussion about budgeting, planning, and evaluation in the 
context of statewide learning goals and evidence of student learning. In the short-term, governors 
can build incentive funding pools to provide new resources to institutions that show the most 
progress in improving learning consistent with state goals. Over time, governors need to require 
institutional and legislative leaders to think about total performance and productivity by looking 
at how state subsidies are used to achieve state learning goals. Eunding of the base budget should 
be contingent on state and institutional agreement about goals and measures for learning 
productivity — combining use of resources with assessments of learning results. 

Conclusion 

Historically, the United States has had the best system of higher education in the world. This 
quality has been characterized by a high degree of institutional diversity, decentralized 
governance, and generous levels of funding. But its quality and durability are being challenged 
through a unique combination of rising student demand, changing demography, and constrained 
state resources. Governors and institutional leaders need to work together to find new tools to 
maintain and increase success to ensure every student has an opportunity for access to high 
quality, affordable education. Einding new ways to measure and account for student learning and 
tie learning outcomes to broad state goals is central to this agenda. With the proper gubernatorial 
leadership, this can be accomplished while maintaining respect for historic values of institutional 
diversity and autonomy. 
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